# Low-resource inference

DiffuCoder-7B-cpGRPO-4bit is a 4-bit quantized version converted from the Apple DiffuCoder-7B-cpGRPO model, optimized for the MLX framework.

A quantized version of Kimi-Dev-72B, using advanced nonlinear optimal quantization and multi-head latent attention mechanism to reduce storage and computing requirements.

A quantized version of the Austral-24B-Winton model of Delta-Vector, quantized using the llama.cpp tool, suitable for efficient operation on different hardware configurations.

This is a 4-bit quantized version converted from the Qwen3-235B-A22B-8bit model, optimized for the MLX framework and suitable for text generation tasks.

This is a project that directly converts bytedance-research/Phantom to the GGUF format for image-to-video conversion tasks.

An 8-bit quantized version based on the DeepSeek-R1-0528-Qwen3-8B model, optimized for Apple Silicon chips and suitable for text generation tasks.

A multilingual text generation model fine-tuned based on unsloth/Llama-3.3-70B-Instruct-bnb-4bit, supporting English, Spanish, Latin, Arabic, and French.

A multilingual text generation model based on Mistral-Small-3.1-24B-Base-2503, supporting 10 languages, suitable for role-playing and dialogue scenarios

Quantized version optimized for low-end hardware and CPU-only environments, achieving production-ready inference configurations under resource constraints

Bielik-4.5B-v3.0-Instruct-GGUF is a Polish large language model released by SpeakLeash, converted from Bielik-4.5B-v3.0-Instruct to GGUF quantized format, suitable for local inference.

An instruction fine-tuned model based on the Llama-3-3B architecture, supporting tasks such as dialogue, reasoning, and role-playing, suitable for general artificial intelligence assistance scenarios.

Ablated version of LLaMA-3-8B-Instruct, using specific techniques to force the model to respond in Turkish

Zero-Mistral-24B is a large language model based on the Mistral architecture, supporting Russian and English, suitable for dialogue and text generation tasks.

An 8-bit quantized text-to-speech model fine-tuned from Canopy Labs' pre-trained model, supporting 24kHz English audio generation

A quantized version based on Google Gemma 3's 27-billion parameter instruction-tuned model, generated using quantization-aware training (QAT) weights, supporting multiple quantization levels to meet different hardware requirements.

This model is an INT4 quantized version of google/gemma-3-12b-it, using the GPTQ algorithm to reduce parameters from 16-bit to 4-bit, significantly decreasing disk space and GPU memory requirements.

This is a GGUF format model converted from the canopylabs/3b-hi-ft-research_release model, supporting Hindi text processing.

This is a Turkish function calling model fine-tuned based on the Turkish-Llama-8b-DPO-v0.1 model, specifically designed for executing Turkish function calling tasks.

This is a quantized version of Google Gemma 3B model, optimized based on llama.cpp, suitable for running in resource-limited environments.

This is the 4-bit quantized version of the Qwen/QwQ-32B model, optimized using the BitsAndBytes library, suitable for text generation tasks in resource-constrained environments.

The RWKV-7 g1 model based on the Flash linear attention mechanism, supporting multilingual processing and having deep thinking ability

An intermediate model during the Tantum merging process, created by merging multiple 24B-parameter Mistral and Llama3 variants, suitable for role-playing and text generation tasks.

This is a GGUF-format converted 8B-parameter language model specialized for fantasy fiction, supporting 128K context length.

This model is converted from HKUST-Audio/Llasa-1B into GGUF format, primarily designed for text-to-speech tasks.

A hybrid model based on multiple 12B parameter models, specializing in Russian and English role-playing and text generation

Large language model fine-tuned based on BitNet 1.58b architecture, with Llama-3-8B-Instruct as the base model, employing extreme quantization techniques

This is the GGUF quantized version of the Polish large language model Bielik-11B-v2.3-Instruct developed by SpeakLeash, suitable for local deployment and use.

This model was converted from microsoft/Phi-3-mini-4k-instruct to GGUF format using llama.cpp via ggml.ai's GGUF-my-repo space.

Llama-3.1-Storm-8B is a model developed based on Llama-3.1-8B-Instruct, aiming to improve the dialogue and function call capabilities of models with 8 billion parameters.

A fine-tuned version of the Llama3.1 8B large language model optimized for Turkish, trained on high-quality Turkish instruction datasets

This is a GGUF format model converted from Google's Gemma 2B model, suitable for text generation tasks.

BitNet b1.58 is a large language model with 1.58-bit quantization. It reduces the computational resource requirements by lowering the weight precision while maintaining performance close to that of a full-precision model.

A fine-tuned version of the Llama3 8b large language model optimized for Turkish, trained on high-quality Turkish instruction datasets

An Italian large language model optimized based on Meta-Llama-3-8B, supporting English and Italian text generation tasks

This is a 4-bit quantized GPTQ model based on Meta Llama 3, quantized by Astronomer, capable of efficient operation on low-VRAM devices.

This is a distilled Cantonese speech recognition model based on Whisper Small, achieving a CER of 9.7 (without punctuation) on Common Voice 16.0.

Multilingual instruction model fine-tuned on Gemma-2b, supporting 15 Indian languages and English

Yugo55A-GPT is a Serbian-optimized large language model merged from multiple excellent models, demonstrating outstanding performance in Serbian LLM evaluations.

Minueza-32M-Base is a base model with 32 million parameters, fully trained on extensive English text corpora, suitable for text generation tasks.

Law LLM 13B is a specific domain foundation model developed based on LLaMA-1-13B, focusing on tasks in the legal domain.